<html>
<head><meta charset="utf-8"><title>contention debugging · t-compiler/wg-parallel-rustc · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/index.html">t-compiler/wg-parallel-rustc</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html">contention debugging</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="181975908"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181975908" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181975908">(Nov 26 2019 at 23:46)</a>:</h4>
<p>I tried to do some loose profiling of locks and such but that mostly failed (wasn't able to get low enough overhead / enough samples)</p>



<a name="181975925"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181975925" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181975925">(Nov 26 2019 at 23:47)</a>:</h4>
<p>mostly pointed at <code>env::var</code> calls in MIR interpret and Cargo, along with Cargo's interner</p>



<a name="181975940"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181975940" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181975940">(Nov 26 2019 at 23:47)</a>:</h4>
<p>neither of which seemed... plausible to actually be a hot spot</p>



<a name="181975952"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181975952" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181975952">(Nov 26 2019 at 23:47)</a>:</h4>
<p>hopefully Santiago will have more success</p>



<a name="181983769"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181983769" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181983769">(Nov 27 2019 at 02:20)</a>:</h4>
<p>Hm well env accesses are actually locked</p>



<a name="181983773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181983773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181983773">(Nov 27 2019 at 02:20)</a>:</h4>
<p>So not entirely implausible if something is slamming env vars</p>



<a name="181983791"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181983791" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181983791">(Nov 27 2019 at 02:21)</a>:</h4>
<p>We may need to instrument rustc to try to learn about contended locks</p>



<a name="181984065"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181984065" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181984065">(Nov 27 2019 at 02:28)</a>:</h4>
<p><a href="https://hackmd.io/CBdNMYStS_GH2MawrgARfw?view#lock-contention" target="_blank" title="https://hackmd.io/CBdNMYStS_GH2MawrgARfw?view#lock-contention">https://hackmd.io/CBdNMYStS_GH2MawrgARfw?view#lock-contention</a></p>



<a name="181984075"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181984075" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181984075">(Nov 27 2019 at 02:28)</a>:</h4>
<p>I think this pretty much does not see parking_lot locks at all, which might explain high percetage here (though unclear -- the numbers from perf stat indicate that it sees all calls to futex, at least)</p>



<a name="181984094"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181984094" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181984094">(Nov 27 2019 at 02:29)</a>:</h4>
<p><code>cargo::core::interning::InternedString::new</code>, <code>&lt;rustc::mir::interpret::error::InterpErrorInfo as core::convert::From&lt;...&gt;&gt;</code>, <code>cargo::util::profile::enabled_level</code></p>



<a name="181984151"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181984151" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181984151">(Nov 27 2019 at 02:31)</a>:</h4>
<p>I think the cargo ones probably don't matter in practice</p>



<a name="181984159"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181984159" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181984159">(Nov 27 2019 at 02:32)</a>:</h4>
<p>like, maybe we could get wins, but ultimately that's not important, cargo is mostly idle</p>



<a name="181984215"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181984215" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181984215">(Nov 27 2019 at 02:33)</a>:</h4>
<p><code>&lt;rustc::mir::interpret::error::InterpErrorInfo as core::convert::From&lt;...&gt;&gt;</code> I plan to file a PR switching it out for a lazy static or something like that</p>



<a name="181984445"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181984445" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181984445">(Nov 27 2019 at 02:38)</a>:</h4>
<p>unfortunately our normal benchmarks (perf.rlo) probably won't notice, as the uncontended lock is presumably quite cheap</p>



<a name="181986152"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181986152" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181986152">(Nov 27 2019 at 03:23)</a>:</h4>
<p>Cargo should be easy to fix yeah, the string one seems the most plausible</p>



<a name="181986154"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/181986154" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#181986154">(Nov 27 2019 at 03:23)</a>:</h4>
<p>Can you swap out parking lot for libstd to get contention numbers?</p>



<a name="182016672"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182016672" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182016672">(Nov 27 2019 at 13:07)</a>:</h4>
<p>hm, maybe, I can try that</p>



<a name="182017038"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182017038" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182017038">(Nov 27 2019 at 13:11)</a>:</h4>
<p>std's APIs are different I think and sufficiently so that this might not be readily possible</p>



<a name="182023563"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182023563" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182023563">(Nov 27 2019 at 14:29)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> also -- forgot to note this -- you'll want to run <code>sudo rm -f /dev/shm/sem.jobserver-rust*</code> between builds; obviously that's not tenable long-term but for now I believe is necessary-ish</p>



<a name="182026202"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182026202" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182026202">(Nov 27 2019 at 14:55)</a>:</h4>
<p><code>sync.rs</code> has a custom <code>Lock</code> wrapper, so API doesn't matter too much (<code>LockGuard</code> is reused though)</p>



<a name="182026390"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182026390" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182026390">(Nov 27 2019 at 14:57)</a>:</h4>
<p>parking lot parks ~2000 times total in the first 3 seconds from raw_mutex...</p>



<a name="182026952"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182026952" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182026952">(Nov 27 2019 at 15:02)</a>:</h4>
<p>~6000 total futex calls from parking lot</p>



<a name="182026956"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182026956" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182026956">(Nov 27 2019 at 15:02)</a>:</h4>
<p>so parking lot is not the source of trouble</p>



<a name="182026972"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182026972" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182026972">(Nov 27 2019 at 15:02)</a>:</h4>
<p>(this is on a run with perf stat reporting 457,198 futex calls)</p>



<a name="182026997"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182026997" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182026997">(Nov 27 2019 at 15:03)</a>:</h4>
<p><span class="user-mention" data-user-id="116466">@Zoxc</span> yeah, std RwLock/Mutex don't support mapping afaict</p>



<a name="182041807"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182041807" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182041807">(Nov 27 2019 at 17:42)</a>:</h4>
<p>initial findings suggest that contention is happening in:</p>
<ul>
<li>(100s of milliseconds - 1.5 seconds(!)) <code>rustc_rayon_core::sleep::Sleep::wake_specific_thread</code> </li>
<li>(100s of nanoseconds) jobserver helper thread (Condvar/Mutex around a usize, fairly easily replaceable)</li>
</ul>



<a name="182041953"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182041953" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182041953">(Nov 27 2019 at 17:44)</a>:</h4>
<p>utilized this patch to libstd to gather data: <a href="https://gist.github.com/Mark-Simulacrum/6dfa7678f2d449175aa1f3d8856340f7" target="_blank" title="https://gist.github.com/Mark-Simulacrum/6dfa7678f2d449175aa1f3d8856340f7">https://gist.github.com/Mark-Simulacrum/6dfa7678f2d449175aa1f3d8856340f7</a></p>



<a name="182041971"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182041971" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182041971">(Nov 27 2019 at 17:44)</a>:</h4>
<p>it has fairly high overhead, though, so measurements may not be super reliable</p>



<a name="182042200"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182042200" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182042200">(Nov 27 2019 at 17:46)</a>:</h4>
<p>overhead is much lower if I adjust the elapsed higher</p>



<a name="182042218"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182042218" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182042218">(Nov 27 2019 at 17:46)</a>:</h4>
<p>(initial measurements were with ~100 nanoseconds)</p>



<a name="182042255"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182042255" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182042255">(Nov 27 2019 at 17:47)</a>:</h4>
<p>but I can get ~65 job units in the 3 second run if I adjust to 300, which is about the same as I get without this</p>



<a name="182042624"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182042624" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182042624">(Nov 27 2019 at 17:51)</a>:</h4>
<p>trying to run some more benchmarks and get a good sense of how reliable this is</p>



<a name="182043278"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182043278" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182043278">(Nov 27 2019 at 17:59)</a>:</h4>
<p>I am consistently seeing stalls for hundreds of milliseconds in rustc_rayon_core::sleep::Sleep::wake_specific_thread</p>



<a name="182043286"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182043286" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182043286">(Nov 27 2019 at 17:59)</a>:</h4>
<p>but maybe that's expected?</p>



<a name="182043676"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182043676" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182043676">(Nov 27 2019 at 18:03)</a>:</h4>
<p>e.g., 51.508068ms; 772.124223ms; 773.960212ms; 58.569277ms; 951.101171ms; 51.476288ms; 866.302171ms; 867.230675ms</p>



<a name="182044479"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182044479" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182044479">(Nov 27 2019 at 18:14)</a>:</h4>
<p>trying a different tack -- perf record can give me the mutex pointers and those are always stable in our case so going to log those on creation time, hopefully there's a) not too many mutexes being created and b) creation isn't as such hot enough that it causes problems</p>



<a name="182054884"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182054884" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182054884">(Nov 27 2019 at 20:14)</a>:</h4>
<p>hm, okay, so that sort of didn't succeed - the address that is given to futex is not of the <code>pthread_mutex_t</code> it seems</p>



<a name="182056481"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182056481" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182056481">(Nov 27 2019 at 20:35)</a>:</h4>
<p>it does seem like a lot of the futex calls are not coming from rustc? but this does not jive well with what we're seeing elsewhere, i.e., in alex's <a href="http://foo.rs" target="_blank" title="http://foo.rs">foo.rs</a> tool</p>
<div class="codehilite"><pre><span></span>    808 cargo
 236398 ld
 125675 rustc
</pre></div>



<a name="182056490"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182056490" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182056490">(Nov 27 2019 at 20:35)</a>:</h4>
<p>I wonder if that tool is buggy</p>



<a name="182057193"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182057193" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182057193">(Nov 27 2019 at 20:44)</a>:</h4>
<p>otoh, 125k is still pretty high</p>



<a name="182060649"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182060649" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182060649">(Nov 27 2019 at 21:36)</a>:</h4>
<p>I am consistently getting most futex calls in ld, not rustc</p>



<a name="182060668"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182060668" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182060668">(Nov 27 2019 at 21:36)</a>:</h4>
<p>and total futex call count is still around 400k so I don't think it's because I've made changes</p>



<a name="182060887"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182060887" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182060887">(Nov 27 2019 at 21:39)</a>:</h4>
<p>I am kinda confused about the futex calls from ld as I didn't think that was parallelized -- I wonder if the ld I have is not the normal ld</p>



<a name="182060945"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182060945" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182060945">(Nov 27 2019 at 21:40)</a>:</h4>
<p>ah, indeed, I'm using lld as my default linker it seems</p>



<a name="182072921"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182072921" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182072921">(Nov 28 2019 at 00:54)</a>:</h4>
<p>What in the world is the linker using futexes for</p>



<a name="182072926"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182072926" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182072926">(Nov 28 2019 at 00:54)</a>:</h4>
<p>Isn't it single threaded?</p>



<a name="182072930"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182072930" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182072930">(Nov 28 2019 at 00:55)</a>:</h4>
<p>lld is not, apparently, according to some random googling</p>



<a name="182072941"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182072941" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182072941">(Nov 28 2019 at 00:55)</a>:</h4>
<p>Interesting, so my script should probably change</p>



<a name="182072944"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182072944" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182072944">(Nov 28 2019 at 00:55)</a>:</h4>
<p>And only account for rustc things</p>



<a name="182072954"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182072954" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182072954">(Nov 28 2019 at 00:55)</a>:</h4>
<p>(to be clear, my interpretation of your script was that it tried to do so)</p>



<a name="182072955"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182072955" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182072955">(Nov 28 2019 at 00:55)</a>:</h4>
<p>The linker we don't care too much about and likely just adds noise</p>



<a name="182072959"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182072959" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182072959">(Nov 28 2019 at 00:55)</a>:</h4>
<p>Oh, nvmd then</p>



<a name="182073004"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073004" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073004">(Nov 28 2019 at 00:56)</a>:</h4>
<p>though maybe it failed since it does seem like the numbers I'm getting are heavily inconsistent with the theory that the script was successful at doing so</p>



<a name="182073026"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073026" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073026">(Nov 28 2019 at 00:57)</a>:</h4>
<p>it's also possible that <code>perf trace</code> has different properties than what I'm doing now, which is (I think) more low level</p>



<a name="182073088"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073088" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073088">(Nov 28 2019 at 00:58)</a>:</h4>
<p>right now I've largely been struggling to figure out how to track down callers of e.g. <code>Mutex::new</code> -- collecting backtraces via std::backtrace is apparently way too slow (I get something like an order of magnitude, or two, slowdown)</p>



<a name="182073107"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073107" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073107">(Nov 28 2019 at 00:59)</a>:</h4>
<p>I'm going to move it to just the lock() method or something</p>



<a name="182073183"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073183" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073183">(Nov 28 2019 at 01:00)</a>:</h4>
<p>I'd actually warn against that</p>



<a name="182073184"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073184" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073184">(Nov 28 2019 at 01:00)</a>:</h4>
<p>Or take.the numbers with a grain of salt</p>



<a name="182073191"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073191" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073191">(Nov 28 2019 at 01:00)</a>:</h4>
<p>Backtraces are single threaded and extremely expensive to capture</p>



<a name="182073200"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073200" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073200">(Nov 28 2019 at 01:01)</a>:</h4>
<p>So it may not be a very reliable way to gather contention data that way</p>



<a name="182073218"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073218" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073218">(Nov 28 2019 at 01:01)</a>:</h4>
<p>to be clear the new plan only collects the backtrace once we've already detected at least some contention</p>



<a name="182073221"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073221" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073221">(Nov 28 2019 at 01:01)</a>:</h4>
<p>but yes, I agree that it's likely not perfect</p>



<a name="182073225"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073225" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073225">(Nov 28 2019 at 01:01)</a>:</h4>
<p>(I'm grasping at straws a bit, to be honest)</p>



<a name="182073288"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073288" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073288">(Nov 28 2019 at 01:03)</a>:</h4>
<p>one possible easy win that I should try though I have no numbers to back me up is to try and switch rayon over to parking_lot</p>



<a name="182073313"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073313" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073313">(Nov 28 2019 at 01:03)</a>:</h4>
<p>since at least primitive benchmarks seem to make us believe that parking_lot is much better at both the contended and uncontended cases, and I am seeing at least some evidence pointing at rayon being a cause of contention</p>



<a name="182073362"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182073362" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182073362">(Nov 28 2019 at 01:04)</a>:</h4>
<p>(in particular, parking lot currently barely calls futex)</p>



<a name="182140906"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182140906" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182140906">(Nov 28 2019 at 23:21)</a>:</h4>
<p>parking lot didn't seem to help much</p>



<a name="182142208"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182142208" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182142208">(Nov 29 2019 at 00:01)</a>:</h4>
<p>here's flamegraph of the futex calls in the first 3 seconds of cargo check on cargo for master, the rayon branch, and rayon+semaphores:</p>
<p><a href="/user_uploads/4715/Jz9g2FT-0ugfyTLlDWWiS1EC/futex-rayon.svg" target="_blank" title="futex-rayon.svg">futex-rayon.svg</a> <a href="/user_uploads/4715/--VgbxKWJZiSsG4r8JPKEKnl/futex-master.svg" target="_blank" title="futex-master.svg">futex-master.svg</a> <a href="/user_uploads/4715/AYnG1WarMDHa_kOv522NhTnR/futex-semaphores.svg" target="_blank" title="futex-semaphores.svg">futex-semaphores.svg</a></p>



<a name="182390347"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182390347" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182390347">(Dec 02 2019 at 21:13)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> <code>sudo --preserve-env perf record -m256 --switch-output=600M --aio=4 -e syscalls:sys_enter_futex --call-graph dwarf -o ~/perf.data -- timeout 3s $CARGO check -j$T</code> with RUSTC and CARGO set appropriately and T=28, then <code>perf script</code> into stack-collapse / flamegraph (I use inferno, but shouldn't matter)</p>



<a name="182390385"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182390385" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182390385">(Dec 02 2019 at 21:13)</a>:</h4>
<p>(I also have my own custom perf script that's quite a bit faster, but I haven't gotten around to publishing it -- it should be about equivalent in terms of output though)</p>



<a name="182390386"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182390386" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182390386">(Dec 02 2019 at 21:13)</a>:</h4>
<p>woo thanks!</p>



<a name="182390407"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/contention%20debugging/near/182390407" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/contention.20debugging.html#182390407">(Dec 02 2019 at 21:13)</a>:</h4>
<p>I'll try and type up the other commands I was using</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>